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This paper addresses the problem of verifying adaptive control techniques for enabling 
safe flight in the presence of adverse conditions. Since the adaptive systems are non- 
linear by design, the existing control verification metrics are not applicable to adaptive 
controllers. Moreover, these systems are in general highly uncertain. Hence, the system’s 
characteristics cannot be evaluated by relying on the available dynamical models. This 
necessitates the development of control verification metrics based on the system’s input- 
output information. For this point of view, a set of metrics is introduced that compares 
the uncertain aircraft’s input-output behavior under the action of an adaptive controller to 
that of a closed-loop linear reference model to be followed by the aircraft. This reference 
model is constructed for each specific maneuver using the exact aerodynamic and mass 
properties of the aircraft to meet the stability and performance requirements commonly 
accepted in flight control. The proposed metrics are unified in the sense that they are 
model independent and not restricted to any specific adaptive control methods. As an 
example, we present simulation results for a wing damaged generic transport aircraft with 
several existing adaptive controllers. 

I. Introduction 

The Aviation Safety Program under the Aeronautics Research Mission Directorate (ARMD) at NASA 
has created the Integrated Resilient Aircraft Control (IRAC) research project to advance the state of aircraft 
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flight control to provide on-board control resilience for ensuring safe flight in the presence of adverse condi- 
tions. 1 The goal of the IRAC project is to arrive at a set of validated multidisciplinary integrated aircraft 
control design tools and techniques for enabling safe flight in the presence of adverse conditions (ex: faults, 
damage and/or upsets). 

One objective towards this goal is to advance the state-of-the-art of adaptive controls as a design option 
to provide enhanced stability and maneuverability margins for safe landing. Adverse events include loss of 
control caused by environmental factors, actuator and sensor faults or failures, and expand toward more 
complicated damage conditions. The application focus of this technology is for current and next generation 
subsonic civil transports. However, a majority of the challenges addressed by the IRAC project are general 
in nature, and therefore, the solutions will apply to a large class of aviation vehicles. 

Furthermore, the IRAC project is also developing system-level adaptive control architectures that can 
effectively deal with various integrated effects such as aero-servo-elastic interaction with flight control and 
integrated aero-propulsion flight control using engine differential thrust. The integration of the fundamental 
adaptive control research into system-level flight control architectures is an important step towards the 
validation and maturation of adaptive control technology. 

In a general sense, a control design for an adverse event can be stated as follows. Given the airplane 
mathematical model, 


x(t) = f (x(t),u(t),r](9(t),A(t),t),t) (1) 

the flight and engine controller need to arrive at a control, u(t) such that the system (in the order of priority) 

- is locally stable 

- follows closely the desired trajectories (includes handling quality of the system) 

- reacts to changing environments by properly adapting the planning functionality. 

In Equation (1), x(t) € R" is the state vector, u(t ) € R m is the control vector, r](9(t), A(i), t) is the 

unknown change caused by the adverse event. The change could include both parametric uncertainties given 
by the finite-dimensional vector function ( 9(t )) and unmodeled uncertainties (A(t)), with /.t = sup ||A(t)|| as 
the upper bound of the unmodeled uncertainties A (£). 

The control u(t) can be further broken down into its subcomponents as follows: 

u(t) = u 0 {x(t),t) + Uad {w(t),p,x(t),5, T) + u c ( 9(t),x(t),x g (t),p(t),t ) , (2) 

where Uo (t) is the nominal control, u a d(t) is the adaptive inner-loop control, u c (t) is the commanded outer- 
loop control, w(t) is the control parameter vector that is updated online, S(t) is the vector of commanded 
signals, T represents adaptation gains and filter parameters, x g (t) is the trajectory guidance function, and 
p(t) is the emergency planning function. 

The foundational research in the IRAC project is focused on arriving at u a d(t) and u c (t) for handling 
a myriad of adverse events. To do this successfully, we need advances in (1) control methodologies; (2) 
modeling methodologies for controller synthesis and validation; and (3) methodologies for verifying and 
validating the control and modeling technologies developed. 

Recent adaptive control approaches have shown great promise in handling certain classes of adverse 
conditions. 2-11 Uncertainty due to an adverse event is a function of both model-parametric uncertainty and 
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unmodeled uncertainty. For example, uncertainties in Cl , Co, etc will fall under the first category. These 
uncertainties can be represented by the set H{9, A), where 9 is the parametric space and A denotes bounds 
on this uncertainty (spherical, rectangular, etc.). Good flight dynamic models will provide an estimate for 
the set H(9, A), and adaptive control laws that will accommodate these uncertainties can be derived. This 
problem is complicated by the presence of unmodeled uncertainty. For the uncertainty vector rj(9(t), A (t),t), 
the relationship between the geometry of H{9 1 A), characterization of /z, and bounds on A (f) is a foundational 
research topic. The current approaches for adaptive control do not provide a precise characterization of /z 
that can in turn be related to preferred stability and performance margins. There have been some preliminary 
results in relating the adaptive controller free parameters to an optimal /z value 12 and to the stability of the 
closed-loop system. 13 

In spite of the advances in adaptive control research and the potential benefits of adaptive systems 
to enhance flight safety in the presence of adverse conditions, the absence of verification and validation 
methods for adaptive control systems remains a major hurdle to the implementation of adaptive control 
in safety-critical systems. 14-16 This hurdle can be traced to the lack of performance and stability metrics 
for adaptive control which poses a major challenge that prevents adaptive control from being implemented 
in safety critical systems. The development of verifiable metrics for adaptive control will be important in 
order to mature adaptive control technology for use in operational safety critical systems. Stability metrics of 
adaptive control are an important consideration for assessing the system’s robustness to unmodeled dynamics 
and exogenous disturbances. Therefore, it is imperative to define a proper set of stability and performance 
metrics for adaptive systems as a first step in the development of reliable verification and validation methods 
leading to adaptive control software certification. 

Another benefit of metrics for adaptive control is in the realization of the need for metrics-driven adaptive 
control. Metrics-driven adaptive control is the notion that adaptation in certain cases should be driven by 
the need to trade-off between stability and performance to maintain safe flight. In Reference, 11 the authors 
have started investigating the feasibility of using a bounded linear stability analysis method to change the 
adaptation rates of the controller on-line to improve the stability margin of the closed-loop system. 

Generally, the existing classical gain and phase margins are not applicable to adaptive control systems, 
because they are nonlinear and time- variant. Moreover, since the adaptive controllers are intended to operate 
through wide operating conditions with fast transitions between them, linerization or frozen-time based 
control methods cannot be effective. Recently some results appeared in the literature on the metrics definition 
and evaluation of adaptive systems. Reference 16 proposes to use the parameter sensitivity as a metric for 
the output of neural networks. Reference 17 introduces stability metrics based on the Lyapunov analysis and 
passivity theory. In references 18, 19 optimization based metrics are introduced for evaluating the stability and 
robustness of adaptive systems. Whereas the first method is capable of partial verification of the closed-loop 
system, the second method can asses the stability of the overall closed-loop system. However, it requires the 
knowledge of a proper Lyapunov or storage function that can be evaluated online. This is in general very 
difficult to provide, since the Lyapunov function for the stability analysis always depends on the parameter 
estimation errors, which are not accessible for computations. The third method requires knowledge of the 
bounds for all uncertainties and involves excessive computations for online optimization. 

There are several unaddressed issues related to the implementation of adaptive control technology in 
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flight vehicles to accommodate uncertainty in adverse events. These issues include but are not limited 
to: (1) achievable stability metrics of adaptive control as related to uncertainty bounds; (2) adaptation 
in the presence of static and dynamic saturation of actuators; (3) cross-coupling between longitudinal and 
lateral-directional axes due to failures, damage, and different rates of adaptation in each axis; (4) on-line 
reconfiguration and control reallocation using non-traditional control effectors such as engines; and (5) time- 
scale separation in actuator systems with different time latencies such as conventional control surfaces and 
engines. 

The development of adaptive control metrics requires active participation of the research community 
which includes academia, industry, and government agencies. From this point of view, the proposed metrics 
can be viewed as preliminary and open for discussion, the goal of which would be to produce a set of acceptable 
metrics that the research community can support. This is viewed as an important process towards developing 
acceptance criteria for adaptive control that would lead to a certification path for adaptive systems. 

In this paper, we address the issue of quality by crafting several stability and performance metrics that 
can be used to measure an adaptive controllers ability to achieve safe flight during an adverse event. Since 
stability and performance of adaptive control cannot be analyzed by the traditional phase and gain margins, 
our hope is to arrive at a family of metrics that will provide the basis for eventual validation and adoption 
of adaptive controllers for piloted aircrafts. These metrics have to be based on the signals that are easy to 
measure, be simple enough for online computation yet be rich enough to enable the user to asses the system’s 
stability and performance characteristics. One way to introduce this type of metrics is to compare the output 
and input signals of the adaptive system to those of a linear time invariant reference model, for which the 
stability and performance metrics are well understood. It is well known that the adaptive controllers can 
achieve asymptotic tracking of the output signal in steady state. However, the transient behavior of the 
system’s output and input are of great importance for flight vehicles and need to be addressed as essential 
characteristics of adaptive systems. These considerations make basis for the proposed metrics, which are 
independent of the system’s dynamics. 

In the following sections, we define several stability and performance metrics suitable for both off-line 
and on-line determination of the quality of the adaptive controllers. In subsequent sections we introduce 
metrics evaluation scenarios and use simulations to examine the effectiveness of the proposed metrics. 

II. Stability and Performance Metrics 

Metric evaluation of adaptive control is based on a comparison of the response of an adaptive control 
design to that of a reference model designed to meet certain stability and performance specifications. For a 
flight vehicle operated in its nominal state, a nominal reference model provides ideal stability and performance 
characteristics of the closed-loop system. A nominal ideal controller is designed to enable the non-adaptive 
closed-loop system to follow the reference model, taking into account actuator constraints such as position 
and rate limits. 

In a damaged state, if the damage is severe enough, the nominal reference model may no longer provide 
satisfactory stability and performance characteristics. Thus, it becomes necessary to revise the reference 
model accordingly to take into account changes in the vehicle dynamics and/or actuator dynamics. For 
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the purpose of evaluating adaptive control, an off-nominal reference model will be designed for each specific 
damage scenario to provide the ideal response of the damaged vehicle. An off-nominal ideal controller is 
designed to enable the off-nominal closed-loop system to accurately track the off-nominal reference model 
for a given command. 

In this section, we introduce a set of stability and performance metrics for the adaptive flight control 
system and discuss each metric from the prospective of assessing the systems characteristics. To this end, 
consider an open-loop model of the airplane dynamics (1), assuming that all uncertainties are known. We 
write the model in the following form 

X m (t) = A 0 m x m {t) + B m [f(x m (t),T](0(t), A(t),t),t) + G(x m (t), r)(0(t), A(t), t), t)u m (t)] , (3) 

where the pair (A^,S m ) is controllable, the nonlinearities of system (1) are represented by the functions 
/(•) and G(-), and the matrix G(-) is invertible. For a given smooth and bounded reference command y c (t), 
the control signal 

u m {t) = G~ 1 (x m (t),r](9(t),A(t),t),t)[-K m x m (t) + Ny c (t) - f(t,x m )] (4) 

can be designed such that the closed-loop linear time invariant model 

^m(t) — A rn x rn {h) + B m Ny c (t) , (5) 

where A m = A {) m — B m K m is a Hurwitz matrix, satisfies the conventional flight control stability and per- 
formance specifications. In addition, we assume that the control signal u m (t) in (4) satisfies the actuator 
magnitude and rate limitations. 

A. Stability Metrics 

Research in stability metrics for nonlinear adaptive control is currently being pursued in the research com- 
munity. Methods for analyzing stability margins for adaptive control are still being developed to address 
the needs for certification requirements. The lack of quantitative stability metrics for adaptive control is a 
major challenge that poses as a hindrance to certifying adaptive control for safety-critical systems. 

1. Gain and Phase Margins. 

Traditionally, flight control systems are certified by demonstrating that they meet specifications for stability 
margins among other things. Typically, certification requirements, such as MIL-F-9490D, are often addressed 
in terms of phase and gain margins. These margins are used for LTI control laws to provide robustness or 
safety margins in a control system design as a safeguard from unmodeled effects and uncertainty. 

While the gain margin concept can be extended to adaptive control (see for example 20 and derivations 
below), the phase margin concept does not easily lend itself to adaptive systems due to the nonlinearity in 
adaptive control. Strictly speaking, a phase margin for adaptive control in a global context as in the LTI 
framework is not possible. 

Here we show that the adaptive system has a gain margin no smaller than that of the reference model. 
To this end, let the system dynamics propagate according to the following differential equation: 

x(t) = Ax(t) + b\u(t) , (6) 
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where A £ R raxn is an unknown matrix, b £ R ra is a known constant vector, and A ^ 0 is an unknown 
constant of known sign. 

Direct model reference adaptive control is defined as: 

u(t) = kj(t)x(t) + k r (t)r(t) (7) 

where k x (t) £ R ra , k r (t) £ R. are the adaptation rates defined via the stability proof, yields the following 
closed-loop system dynamics: 

x(t) = (A + bXkJ + bXk r (t)r(t) . (8) 

When the matching conditions 

3 k*, bX(k*) T = A m - A 

3 k*, bXk* = b m (9) 

are satisfied, the dynamics of the tracking error e(f) = x(t) — x m (t) take the form 

e(t) = A m e(t) + bX (A/cJ (t)x(t) + A k r (t)r(t)) , (10) 

where A k x (t) = k x (t ) — k*, A k r (t) = k r (t) — k* denote parameter errors. Defining the adaptation laws as 

kx(t) = —T x x(t)e T (t) P 6 sign( A) i: k x (0) = k x0 , 

k r (t ) = — 7 r r(t)e T (t)P 6 sign(A), fc r (0) = k r o , (11) 

where r x = T J > 0, y r > 0 are the adaptation gain and P = P T > 0 solves the algebraic Lyapunov equation 

Aj„P + PA m = -Q (12) 

for any Q = Q T > 0, results in e(t) — > 0 as t — > oo. Notice that the tracking error converges to zero for any 
positive values of L x and "f r . 

Now we introduce a gain K in the control channel. The system becomes 

x(t) = Ax(t) + KbXu(t ) . (13) 

If the reference model has a gain margin Gm , then for any value of K less than Gm the matrix A = 
A — KB m K m is Hurwitz. Following the same steps, we obtain the tracking error dynamics in the form 

e(t) = Ae(t) + bX (/LAfcJ (t)x(t) + KAk r (t)r(t)) . (14) 

Therefore, for the closed-loop system’s stability the adaptive laws has to be defined as 

Kk x (t ) = — T^a; (t) e T (t) P 6 sign( A) 

Kk r {t ) = — 7 r r(t)e T (t)P 6 sign(A) , (15) 

which coincide with the adaptive laws in (11) with T x and 7 r replaced with T x /K and 7 r /K. That is, the 
closed-loop adaptive system has the same gain margin as the reference model if the matching conditions are 
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satisfied. Notice that the gain margin I\ can be absorbed into adaptation rates and q r by introducing 
notations = K~ 1 T X and q r = K~ l j r . Therefore, the gain margin for the adaptive system does not have 
the same meaning as for the LTI system. 

However, phase and gain margins can be used as stability metrics in adaptive systems under some 
circumstances. One possible use would be when an adaptive control process is terminated by switching off 
the adaptation, essentially freezing the weights, or when the adaptive signal converges to a steady state 
value. 

The stability metrics using the phase and gain margins, denoted by M\ and M 2 metrics where 

Mi - GM 

M 2 = PM, (16) 

are evaluated for an open-loop transfer function of the (linearized if necessary ) system between at the 
system’s input or the output break point at the time instance when the adaptation is switched off. 

Some methods of approximate phase and gain margin analysis for adaptive control have been proposed 
that could be used without turning off the adaptation. Two such approaches are proposed in References. 12, 13 
Whereas in the former it is proposed to define an LTI system asymptotically close to the closed-loop adaptive 
system and evaluate the phase margin for the asymptotic system hoping that the actual adaptive system 
will have a similar behavior, in the latter it is proposed to linearize a system locally on some time window 
and evaluate the phase margin for the linearized system. However, further theoretical justifications for these 
approximate methods are needed. 

2. Time-delay Margin 

Time delay margin is a suitable stability metric for adaptive control. It is defined as the time delay td in the 
control signal that the closed-loop system can tolerate without instability. Time delay can also be introduced 
at other points in a control system besides at the input to the plant. For each time delay implementation, 
there is an associated time delay margin that can be quite different even for the same closed-loop system. 

While time delay margin is a suitable stability metric for adaptive control, a current challenge is that 
there is no well-established analytical tool for computing the time delay margin. Recently, a lower bound on 
the time delay margin has been established for the L\ adaptive control. 21 Other methods for estimating the 
time delay margin have been proposed. One such method applies a Pade approximation to approximate a 
time-delay system. 22 The Pade approximation transforms the original time-delay system into a higher order 
system without the time delay that can be analyzed by the Lyapunov method to estimate the time delay 
margin. Another method has been proposed to analyze the time delay margin using the Razumikhin method 
without numerical approximation of the time delay effect. 23 

Despite the fact that new theoretical methods are being developed for computing time delay margin, they 
are still not ready to be used in a unified framework like the traditional phase and gain margins. The most 
direct way to compute time delay margin is by simulations. The time delay td is introduced into an adaptive 
control design and then adjusted until the closed-loop system is on the verge of instability. For the purpose 
of quantifying how close the system is to the instability, a specified level of time-delay system response 
relative to the nominal system response without time delay can be selected as a threshold of instability. For 
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example, the threshold of instability could be defined as the level of the time-delay system response at twice 
the nominal system response. Once the threshold of instability is established, then the time delay metric is 
defined as 

M 3 = TDM = t d , (17) 

where td is obtained by adjusting the time delay until the Coo norm of one of the system states exceeds the 
threshold of instability. Because the time delay margin is estimated by numerical evidence from simulations, 
it is important to use the same time step in simulations when evaluating different adaptive control methods. 


B. Performance metrics 

1. Transient Performance metrics 

Transient response is an important requirement for any control system design. In an LTI system, classical 
metrics of overshoot or undershoot are used to bound the exceedance of the transient response of a control 
system to a specified command. With a large transient response, the system may move to a different 
operating point that could invalidate the control system design. A transient performance metric measures 
the overshoot or undershoot of the response of an adaptive system relative to the ideal response of the 
off-nominal reference model to a specified command. However, there is some ambiguity in separating the 
transient and steady-state. We want to point out that the settling time computed for the reference model 
is not necessarily a settling time for the adaptive system. In fact, the settling time for the adaptive system 
may be computed by evaluating the performance metrics. 

One transient performance metric is the C Q 0 norm of the tracking error relative to the off-nominal reference 
model and is defined as 


M A = \\x(t)-x rn {t)\\c 00 = max ||a:(t) - x m (t) || , 

te[t 0 ,t f ] 


(18) 


where x(t) is the state of the adaptive system (1), x m (t) is the state of the reference model (1), ||u|| is the 
Euclidean norm of vector v, to is the initial time at which an adverse event is introduced and tf is the final 
time to be determined. This could be the final simulation time or the settling time at which the response 
of an adaptive system is within a specified tolerance of the reference model and never leaves it. However, 
taking into account the nonlinearity of adaptive systems and possible unwanted phenomena, like bursting, 
in some cases the settling time can be determined only after the simulation is complete. We notice that the 
metrics M 4 can be evaluated for each component of the system’s state. 

For a flight control system, this metric can be normalized to produce a value that scales with different 
outputs of the reference model. The normalized transient performance metric is defined as 


\\x(t) ~ XmjtfWc^ 

IkmWIUoo 


(19) 


Another possible transient performance metric is the C 2 norm of the tracking error relative to the off- 
nominal reference model evaluated for a specified time duration 


M 5 




%m(t ) || £2 


A 



- x m (f )\\ 2 dt. 


(20) 
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This metric measures oscillations of the response of an adaptive system relative to the ideal response of 
the reference model. Since an oscillatory signal will result in a larger C 2 norm, this metric also measures the 
high frequency response of an adaptive system, which is important from the perspective of preventing the 
aircraft’s structural elements from overloading. 

The M 5 metric can be also evaluated for each state separately as well as as well as in the normalized 
form 


as in the case of M 4 metric. 


M 5 


|| a; m(t) \\c 2 


(21) 


2. Steady-State Performance metrics 

The ability for adaptive control to quickly settle to a steady state operating condition is important for 
evaluating the effectiveness of an adaptive system. A steady state performance metric is designed to provide 
a specification of the time duration for the response of an adaptive system to reach a specified level of the 
expected ideal response of the reference model. This is equivalent to the settling time in classical control. 
So one of the possible steady-state metrics can be the settling time of an adaptive system. However, as 
mentioned above, in some cases the settling time can only be determined after the simulation is complete. 

Other steady-state performance metrics can be M 4 and M 5 metrics evaluated on a time interval corre- 
sponding to a steady-state behavior of the adaptive system. In fact, in simulation M 4 and M 5 metrics can be 
continuously evaluated online, and the steady-state behavior can be determined based on those evaluations. 


3. Control Signal Metrics 


The ability to achieve control objectives that satisfy actuator constraints should be evaluated for different 
adaptive control design methods. With position or rate saturation, the tendency for pilot-induced oscillations 
(PIO) increases. An ideal controller is one that provides dynamically achievable actuator commands which 
satisfy both position and rate limits. Control limits could be measured by the Coo and C 2 norms of the 
control signal error relative to the off-nominal ideal controller. 

One proposed control limiting metric is as follows: 

M 6 = ||u(f) - UmitfWc^ = max ||u(f) - u m (t)|| , ( 22 ) 

where u(t) is the adaptive control signal (2), u m (t) is the desired ideal control signal given by (4) and designed 
to provide dynamically achievable commands to the actuator systems. The normalized version of this metric 
is 


Mr = 


I u(t) - U m (t)\\c o 


(23) 


lkm(t)||,Coc 

This metric measures the overshoot of the adaptive control signal relative to the ideal control signal. It also 
provides a measure of actuator position limit saturation. 

The next proposed control limiting metric is defined as 


M 7 = || u(t) - u m {t)\\c 2 = \\u(t) - u m (t )\\ 2 dt, 


(24) 
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or when normalized 


M 7 = 


( 25 ) 


\u(t) - u m (t)\\c 2 

|| (t) || C2 

This metric measures the oscillations of the adaptive control signal relative to the ideal control signal of the 
reference model. It provides a measure of the excessiveness of the actuator usage, as well as a measure of 
both actuator position and rate limits. 


C. Cross-Coupling Metrics 

Flight dynamic cross coupling is normally caused by asymmetric failures and/or damage. Pitch-roll coupling 
exists when the roll response is significant when only a pilot longitudinal stick command is given. Cross- 
coupling can also exist when an adaptive system uses cross-coupled product terms in the adaptive law. Cross 
coupling can produce objectionable handling qualities from the pilot perspective, leading to a PIO tendency. 
There are a number of possible metrics for cross coupling. One proposed cross-coupling metric is defined as 


M« = 



5%(t) (p 2 (t) + r 2 (t))dt, 


( 26 ) 


where 5 e (t) is the elevator surface command, and p{t) and r(t) are the roll and yaw rates, respectively. This 
metric measures the pitch coupling response of a flight control system. In an ideal situation, an aircraft 
should only respond in the pitch axis to a pilot longitudinal stick command. Any pitch coupling will result 
in responses in the roll and yaw axes which can be captured by this metric. 

Another proposed cross coupling metric is 


M q = 



q 2 (t) (5 2 (t) + 6 2 (t))dt. 


( 27 ) 


This metric is a measure of the cross coupling in the roll and yaw axes with the pitch axis. In an ideal 
situation, when a pilot applies lateral stick and rudder pedal inputs, very little pitch response should result. 
When a cross coupling is present, the aircraft will exhibit a pitch response which will increase the magnitude 
of this metric. 

Another proposed cross coupling metric that collectively takes into account all angular rate responses is 
the following C 2 norm metric 


M10 — 



\p e (t)q e {t) + p e (t)r e {t) + q e (t)r e (t)] 2 dt , 


( 28 ) 


where p e (t) = pit) — p m (t), q e (t) = q(t) — q m (t), and r e (t) = r(t) — r m (t) are the rate errors. This metric 
measures all the cross coupling effects in all the three control axes relative to the ideal response of the 
reference model. In an ideal situation, the reference model will produce a pure pitch response due to a 
longitudinal stick input. The error in the cross coupled product terms between the response of a flight 
control system and that of a reference model measures the magnitude of the cross coupling. 

D. Handling Quality Metrics 

The presence of a pilot working in a closed-loop fashion with an adaptive flight control system presents an 
important problem that has not been well addressed. This problem is related to the potential interaction 
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between two adaptive elements in a closed-loop fashion, because the pilot can also be viewed as an adaptive 
control system with a learning ability. During adaptation under off-nominal conditions such as aircraft with 
damage, the pilot can introduce excitation signals into the adaptive control system with a predetermined 
adaptation rate. This can degrade the closed-loop performance, generate oscillations and even lead to 
instability. Moreover, the resulting oscillations can potentially cause significant cross-coupling between 
different flight control axes and or excite unmodeled dynamics such as aero-servo-elastic modes. To further 
compound the problem, there is a variance in pilot behaviors that can affect the handling qualities. Pilots 
interacting with adaptive control in commercial transport aircrafts would likely exhibit different behaviors 
than those in military aircrafts. Because of pilot adaptability, the overall performance of a flight control 
system could be significantly influenced by the pilot as opposed to being primarily determined by an adaptive 
flight control system. 

Handling quality metrics such as Cooper-Harper Rating (CHR) are pilot-assessed metrics used to evaluate 
a flight control system performance in the presence of a pilot in the loop. Quantitative measures of CHR in 
desktop simulations are difficult to establish unless a pilot interaction model is available. Flight simulators 
can be used to conduct pilot-in-the-loop handling quality studies to evaluate pilot interaction with an adaptive 
flight control system. To obtain meaningful handling quality rating, the simulation model should include 
sufficient measures of system uncertainty such as flexible aircraft dynamics, atmospheric disturbances, system 
latency, etc. Handling quality studies may be best conducted in a full-scale flight vehicle when all real physical 
effects and system constraints come to bear on a fight control system. In this environment, any deficiency 
in the stability and performance of an adaptive flight control system will likely be discernible by the pilot 
and reflected in the CHR. To obtain any meaningful handling quality metrics, a sufficient sample of pilots 
should be made available in the study to capture the variance in pilot behaviors. 

E. Structural Load Metrics 

An aircraft is designed to meet certain design load factor requirements for safety and passenger comfort such 
as those specified in FAR 25 ” Airworthiness Standards : TransportCategoryAirplan.es " . Load factor is 
the ratio of lift to weight and is a measure of acceleration. During cruise, when the load factor is in excess of 
unity, a vertical acceleration will be produced. Similarly, during a horizontal turn, the load factor is greater 
than unity and produces a lateral acceleration. An adaptive flight control system can potentially produce 
a high bandwidth control signal that can cause load exceedance on a flight vehicle. Structural load metrics 
can be established by the C 0 0 norms of the vertical and lateral accelerations as follows: 

Mu = ||n z (t)|| 00 = max \n z {t)\ (29) 

M 12 = ||%(i)||oo = max \n y (t)\. (30) 

te[to,tf] 

III. Metric Evaluation Process 

In this section, we define a simulation model as a common benchmark for evaluation of the adaptive 
controllers in various failure scenarios for several specific flight phases. 
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A. The Simulation Model 


Many different adaptive control design methods have their inherent strengths and weaknesses. Control design 
is a trade-off process that balances all important design requirements such as stability and performance. In 
order to compare relative merits of different adaptive control features, it is necessary to establish a framework 
for metric evaluation. Towards this end, the IRAC project is to release an open-source model of the sub-scale 
Generic Transport Model (GTM), that will be used to conduct adaptive control studies. 

The model will include a subset of the following damage scenarios including left wing separation, left 
horizontal tail separation, and vertical tail separation, as shown in Table 1 and schematically displayed in 
Figure l 24 


Table 1. Damage conditions. 


Left Wing 

Left Horizontal Stabilizer 

Vertical Tail 

• Tip Loss 4% Semispan 

• Tip Loss 12.5% Semispan 

• Tip Loss 12.5% Semispan 

• Tip Loss 11% Semispan 

• Tip Loss 25% Semispan 

• Tip Loss 25% Semispan 

• Tip Loss 18% Semispan 

• Tip Loss 50% Semispan 

• Tip Loss 50% Semispan 

• Tip Loss 26% Semispan 

• Full Left Horizontal Tail Loss 

• Full Vertical Tail Loss 

• Tip Loss 33% Semispan 

• Left Elevator Loss 

• Rudder Loss 

• Tip Loss 55% Semispan 

• Inboard Flap Loss 

• Outboard Flap Loss 

• Hole Inboard of Engine 

• Hole Outboard of Engine 

• Hole at 40% Semispan 

• Hole at 35% Semispan 


Currently, the GTM model will include a nominal flight controller for the undamaged GTM with a limited 
control allocation capability for split control surfaces. To facilitate metric evaluation, a development process 
is under way to define an ideal reference model for each damage scenario available in the GTM model. These 
ideal damage reference models will be used as benchmarks for comparing different adaptive control design 
methods using an accepted set of metrics. 

The goal of the evaluation is to gain further insight into the relative merits of different adaptive control 
features. Some of these features might be easier to evaluate in a simulated environment and some might 
be better suited for full-scale flight tests. Flight test validation will produce real-world data along with 
pilot handling qualities that will be used to gain a better understanding of the performance of an adaptive 
flight control system and, more importantly, the process of verification and validation of adaptive systems. 
Ultimately, for adaptive systems to be operational in future aviation systems, they must be proven to be 
highly safe and reliable. The development of metrics is a first step towards this goal that can bring about 
an acceptance paradigm for adaptive control. 
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Figure 1. Damage schematics. 


B. Test scenarios 

The possible test scenarios we consider here are straight and level flight, coordinated turning flight, and 
descent or approach flight (See 25 ). After the damage occurs, stabilization to achieve wings-level flight is 
highly desirable. Therefore, it is reasonable to evaluate the proposed metrics for assuming that the aircraft 
was in straight and level flight before the damage, and the objective is to stabilize the aircraft about a trim 
condition, with possibly different trim values, and maintain straight and level flight. 

The next scenario is a coordinated turn, necessary to align the aircraft with a required course. In some 
damage situations, the desired turn direction may not be achievable. For example, with the left wing damage, 
it is more difficult to make a right turn than a left turn. However, for the metrics evaluation purposes, it 
is assumed that the decision making algorithm has knowledge of the specific damage condition, and the 
corresponding reference model is available. Therefore, the proposed metrics can be evaluated for different 
damage conditions from Table 1, using the given reference model. The transition from the straight and level 
reference flight to a turning reference flight may require a re-tuning for some adaptive controllers. 

The final test scenario that we propose is the approach or decent phase of flight. There are two ways to 
evaluate the metrics for adaptive controllers in this scenario. We can assume that the aircraft enters into this 
phase with a prior damage and that the adaptation rates had enough time to move from the initial values, 
or we can assume that the damage occurs during a regular decent maneuver, and the adaptation rates start 
from an initialized set. The second scenario is more challenging, especially for an aircraft with a damaged 
horizontal stabilizer. However, for some adaptive controllers the first scenario may require re-tuning of the 
adaptation rates. 
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IV. Simulation Results 


For the simulation, we use the full-scale generic transport aircraft model (GTM) with a conventional 
model reference adaptive control (MR.AC). The initial flight condition is chosen as straight and level flight 
at the altitude of 15000/f with the speed of M = 0.6. The damage occurs at the initial time and is 
represented by the initial perturbations of the a/c states, which are given in the Table 2. We consider 
damage corresponding to the 26% loss of left wing tip from Table 1. The flight trajectory flown contains the 
flight phases described in the previous section. The aircraft is recovered and holds a straight and level flight 
for 20 seconds. Then a turn is initiated with a bank angle of 15°, lasting 20 seconds, after which straight and 
level flight is resumed for 10 seconds. The final maneuver performed is a descent with the rate of 10/f/sec 
and lasts 20 seconds. 


Table 2. Initial conditions 



Trim before damage 

Trim after damage 

Out of trim initial conditions 

a [°] 

3.37 

5.87 

7.00 

p [rad/s] 

0.0 

0.0 

0.4 

<M°] 

0.0 

-3.16 

15.00 


We evaluate only inner-loop adaptive control for the damaged GTM, that follows the angular rate com- 
mands filtered through a first order reference model with frequencies u> p = 3.5, to q = 2.5, u> r = 2.0. The 
control design is based on the adaptive augmentation of a baseline proportional-integral dynamic inversion 
controller. The aptive augmentation is the MR.AC with identical gains in all of the channels. 

The metrics that are evaluated are the normalized versions of M 3 , M 4 , M 5 , Mg, M 7 , and M§ from Section 
II given by the equations (17), (19), (21), (23), (25) and (26) respectively. 

We run six simulations with and without measurement noise and input delay as shown in Table 3. 


Table 3. Simulation cases 


Case 

Adaptation rate 

Noise 

Time delay 

1 

50 

No 

0 [s] 

2 

200 

No 

0 [s] 

3 

50 

No 

0.07 [s] 

4 

200 

No 

0.07 [s] 

5 

50 

Yes 

0 [s] 

6 

200 

Yes 

0 [s] 


The results of the simulations with adaptation rates 50 and 200, and without measurement noise and 
input delay are summarized in Table 4, and Figure 2 displays the angular rates tracking performance when 
the adaptation rate is 50. It can be seen from the figure that there is a large transient error attributed to the 
large parameter estimation error in the initial stage, since the adaptive parameters are initialized at zero. 
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Also some error shows up at the beginning of each flight regime, as it normally should be with the MRAC 
design. 


Table 4. Metrics for simulation without noise, without time delay (Case 1 and 2) 



Mg 

m 4 

M 5 

M 6 

M'J 

Mg 

r = so 

0.199 

0.3761 

0.5785 

0.2991 

0.3859 

0.0140 

r = 200 

0.090 

0.4353 

0.5836 

0.4427 

0.5230 

0.0127 


Roll Rate 



Pitch Rate 


Yaw Rate 




Figure 2. Angular rate responses in raidians/second in Case 1. 

As can be seen from the table, the time delay margin decreases when the adaptation rates are increased. 
This was expected. In general with the increase of adaptation rates the tracking error is expected to 
decrease, as accounted by metric M 4 . However Table 4 shows that M 4 is greater for the higher adaptation 
rate. This needs further investigation into the best ways to initialize the controllers to avoid errors caused 
by initialization. The increase in metric Mg is small, indicating that the oscillations in output tracking are 
less sensitive to the adaptation rates for the chosen case. Metrics Mq and M-j are substantially increased 
with the increase of the adaptation rates, indicating that the quality of the control signal both in overshoot 
and oscillations are affected when the adaptation rates are increased. The cross-coupling metric Mg is very 
small and insensitive to the change in the adaptation rate at the considered level mainly because of small 
perturbations in the pitch axis. Figure 3 displays the tracking error norm normalized by the maximum value 
of the reference output norm for the angular rate signals and likewise for the control surface deflection signals. 
They are decreased as the simulation proceeds and the adaptation converges to some suitable value. Figure 
4 displays the Mg and M 7 metrics as evaluated over time. As expected, the output oscillatory behavior does 
not diminish with diminishing input oscillations. Similarly, Figure 4 displays the Mg evaluation showing a 
small amount of cross-coupling in the longitudinal and lateral motions. 

Table 5 displays the response of the system expressed by the metrics evaluations when the input delay is 
inserted. Whereas the output related metrics increase compared to non-delay case, the control signal related 
metrics show improvement. The results also show the same type of behavior as in the previous case except 
for the Mg metric which shows that the output tracking oscillations are decreased with the increase in the 
adaptation rates. 
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Figure 3. Normalized output error norm and input error norm in Case 1. 


Metric M s Metric M 7 




Figure 4. Evaluation of metrics M 5 and M 7 as a function of simulation time in Case 1 


Table 5 displays the metrics evaluation results when the input is corrupted with the noise. The results 
are almost identical with those in no-noise case. It may be attributed to adaptive control capabilities to 
compensate for some level of measurement noise. However, further analysis is required to understand why 
it is so and what is the level of noise that can be tolerated. 

V. Discussions 

The discussed set of metrics is by no means considered to be complete. There are other characteristics 
of adaptive systems that one may desire to quantify by means of some specific metrics. One of these is 
the system’s robustness to measurement noise and external disturbances. Unlike LTI systems, the adaptive 
systems do not possess input-output scaling properties. Therefore, the gain margins or time delay margins 
of the adaptive systems do not directly relate to external disturbances and measurement noise. Currently, 
the available information about the systems’ responses to external disturbances and measurement noise are 
limited to the input and output signals of the systems. That is, currently there is no analytic way to distin- 
guish what portion of the output and input errors is contributed by external disturbances or measurement 
noise. Specific test scenarios could be devised to better address the problem. 

Another possible measure of the adaptive control performance could be the robustness to the initialization 
of controller parameters, such as adaptive estimates, reference model, prediction model etc. It is known 
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Figure 5. Evaluation of the cross-coupling metric as a function of simulation time in Case 1. 


Table 5. Metrics for simulation without noise, with 0.07 seconds time delay (Case 3 and 4) 



m 3 

Mi 

M 5 

Me 

m 7 

M 8 

r = so 

0.129 

0.4632 

0.7078 

0.2753 

0.3703 

0.0164 

r = 200 

0.020 

0.4899 

0.5275 

0.4604 

0.5180 

0.0133 


that the role of parameter initialization for the adaptive process and the role of system initialization have 
different contributions to robustness analysis. Therefore, to distinguish and quantify them, we may need 
special metrics. 

In this paper, we have provided a set of metrics that could be evaluated in simulation. We documented 
some simulation examples for evaluating the proposed metrics for conventional model reference adaptive 
control using a wing damaged generic transport aircraft model as a platform in straight and level, coordinated 
turn, and approach flight scenarios. The simulations show that the proposed metrics could be utilized in the 
assessment of the quality of chosen adaptive control method. More simulation studies and comparisons are 
needed to better understand the relative merits of these metrics. The proposed set of metrics can serve as a 
starting point in the development of a road-map to certification of adaptive systems for safety critical flight 
control. With inputs from the research community, this initial set of metrics can evolve into a more detailed 
set of metrics for evaluation of adaptive control. It is expected that the development of these metrics will 
play an integral role in the future research directions ranging from fundamental adaptive control theory to 
system-level implementation of adaptive control for validation in flight test-beds. 
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Table 6. Metrics for simulation with noise, without time delay (Case 5 and 6) 



m 3 

m 4 

m 5 

M 6 

m 7 

M 8 

r = 50 

0.199 

0.3761 

0.5775 

0.3004 

0.3866 

0.0140 

r = 200 

0.090 

0.4369 

0.5852 

0.4339 

0.5251 

0.0128 
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